"v"} International Journal of Scientific Engineering and Technology 
r Volume No.3 Issue No .4, pp : 375-379 



IJ5ET 



(ISSN : 2277-1581) 
1 April 2014 



Comparative Study of Different Image fusion Techniques 

Ms. Mukta V. Parvatikar 1 and Prof. Gargi S. Phadke 2 

^-tech student, Department of Instrumentation, R.A.I.T, Mumbai University, India 
Asst. Prof, Department of Instrumentation, R.A.I.T, Mumbai University, India 

i fj 

Corresponding Email: mparvatikar@gmail.com, gargi.phadke@gmail.com 



Abstract 

Image Fusion is one of the major research fields in image 
processing. It is a process of combining the relevant 
information from a set of images, into a single image, 
without the introduction of distortion wherein the resultant 
fused image will be more informative and complete than any 
of the input images. Image fusion techniques can improve 
the quality and increase the application of these data. This 
paper discusses some of the existing image fusion techniques 
for image fusion like the Averaging Method, Select 
Maximum/Select Minimum, Discrete Wavelet transform 
based fusion and Principal component analysis (PC A) based 
fusion and gives their comparative study together. This 
report also gives evaluation techniques used to evaluate 
fused images along with the applications of image fusion. 

Keywords: Image Fusion, Discrete Wavelet Transform, 
Applications, Image Quality Metrics, Comparison 



I. 



Introduction 



With the recent rapid developments in the field of sensing 
technologies, multisensory systems have become a reality in a 
growing number of fields such as remote sensing, medical 
imaging, machine vision and the military applications for 
which they were first developed. The result of the use of these 
techniques is a great increase in the amount of data available. 
Image fusion provides an effective way of reducing this 
increasing volume of information while at the same time 
extracting all the useful information from the source images. 
The aim of image fusion, apart from reducing the amount of 
data, is to create single enhanced image more suitable for the 
purpose of human visual perception and for further image 
processing tasks such as segmentation, object detection or 
target recognition in applications such as remote sensing and 
medical imaging. 



II. 



Categories Of Image Fusion 



We categorize the IF methods according to the data entering 
the fusion and according to the fusion purpose. Input images 
could be fro many of the following categories [xvii]: 

1) Multimodal Images: Multimodal fusion of images is 
applied to images coming from different modalities like 
visible and infrared, CT and NMR, or panchromatic and 
multispectral satellite images. The goal of the multimodal 
image fusion system is to decrease the amount of data and 
to emphasize band-specific information. 

2) Multifocal Images: In applications of digital cameras, 
when a lens focuses on a subject at a certain distance, all 
subjects at that distance are not sharply focused. A 
possible way to solve this problem is by image fusion, in 
which one can acquire a series of pictures with different 



focus settings and fuse them to produce a single image 
with extended depth of field. The goal of this type of 
fusion is to obtain a single all-in focus image. 

3) Multi-view Images: In multi-view image fusion, a set of 
images of the same scene is taken by the same sensor but 
from different viewpoints or several 3D acquisitions of 
the same specimen taken from different viewpoints are 
fused to obtain an image with higher resolution. The goal 
of this type of fusion is to provide complementary 
information from different viewpoints. 

4) Multi-Temporal Images: In Multi -temporal image fusion, 
images taken at different times (seconds to years) in order 
to detect changes between them are fused together to 
obtain one single image. 

III. Classification Of Image Fusion Techniques 

The actual fusion process can take place at different levels of 
information representation. Image fusion algorithms can be 
categorized into different levels [i]:- 

1) Low or pixel level: The pixel-level method works either 
in the spatial domain or in the transform domain. Image 
fusion at pixel level amounts to integration of low-level 
information, in most cases physical measurements such as 
intensity 

2) Middle or feature level: The feature -level algorithms 
typically segment the image into contiguous regions and 
fuse the regions together using their properties. The 
features used may be calculated separately from each 
image or they may be obtained by the simultaneous 
processing of all the images. 

3) High or decision level: Decision level fusion uses the 
outputs of initial object detection and classification as 
inputs to the fusion algorithm to perform the data 
integration. 

Image fusion method can be broadly classified into two 
groups :- 

a) Spatial domain fusion method: In spatial domain 
techniques, we directly deal with the image pixels. The 
pixel values are manipulated to achieve desired result. 

b) Transform domain fusion method: In transform domain 
method image is first transferred in to frequency domain. 

There are various methods that have been developed to 
perform image fusion. Some well-known image fusion 
methods are listed below [v]: 

1. Intensity -hue-saturation (IHS) transform based fusion. 

2. Principal component analysis (PCA) based fusion. 

3. Arithmetic combinations 

a. Brovey transform. 

b. Synthetic variable ratio technique. 

c. Ratio enhancement technique 

4. Multi scale trans form based fusion 
a. High-pass filtering method. 
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b. Pyramid method 

• Gaussian pyramid. 

• Laplacian Pyramid 

• Gradient pyramid. 

• Morphological pyramid 

• Ratio of low pass pyramid. 

• Co n tras t p y ramid . 

• Filter-subtract-decimate pyramid. 

c. Wavelet transforms 

• Discrete wavelet transforms (DWT). 

• Stationary wavelet transforms 

• Multi-wavelet transforms 

• Dual tree discrete wavelet transforms . 

• Lifting wavelet transform. 

d. Curvelet transforms 

5. Total probability density fusion. 

6. Biologically inspired information fusion. 

IV. Image Fusion Techniques 

A. Simple Average Method 

In this method the resultant fused image is obtained by taking 
the average intensity of corresponding pixels from both the 
input image. The equation used to find the average value is 
given in equation (i) [v] . 



....(1) 



Where X (i, j) and Y (i, j) are two input images. 



B. Select Maximum/ Minimum Method 

A selection process is performed here wherein, for every 

corresponding pixel in the input images, the pixel with 

maximum/minimum intensity is selected, respectively, and is 

put in as the resultant pixel of the fused image [v]. Equation 

(2) and (3) give the maximum and minimum method equation 

respectively. 



ll = D*— l;=D 
v — T — i R 

PU.J) =} } minXQ,fjYQ,f) 
Where X (i, j) and Y (i, j) are two input images. 



....(2) 
.... (3) 



C. Brovey Transform (BT) 

Brovey transform (BT), also known as colour normalized 
fusion, is based on the chromaticity transform and the concept 
of intensity modulation. It is a simple method to merge data 
fromdifferent sensors, which can preserve the relative spectral 
contributions of each pixel but replace its overall brightness 
with the high spatial resolution image [v]. The basic procedure 
of the Brovey Transform first multiplies each MS band by the 
high-resolution PAN band, and then divides each product by 
the sum of the MS bands. The Brovey Transform is good for 
producing RGB images with a higher degree of contrast in the 
low and high ends of the image histogram and for producing 



visually appealing images. Consequently, it should not be used 
if preserving the original scene radiometry is important. 
However, Since the Brovey Transform is intended to produce 
RGB images, only three bands at a time should be merged 
from the input multispectral scene. 

D. Intensity Hue Saturation (IHS) Technique 
It is most popular fusion methods used in remote sensing. The 
fusion is based on the RGB-HIS conversion model, whose 
various mathematical representations have been developed. 
The intensity I defines the total color brightness and exhibits 
as the dominant component [v]. The main steps for the 
standard IHS fusion scheme are:- 

i. Perform image registration (IR) to PAN and MS, and 
res ample MS. 

ii. Convert MS from RGB space into IHS space. 

iii. Match the histogram of PAN to the histogram of the I 
component. 

iv. Replace the I component with PAN. 

v. Convert the fused MS back to RGB space. 

Image 1 (low resolution) 



Image 2 :- high resolution 
Saturation Intensity 




Fused image 



Figure 1: IHS Transformation [xvii] 

E. Principal Component Analysis (PCA) Technique 
Principal Component Analysis is a sub space method, which 
reduces the multidimensional data sets into lower dimensions 
for analysis. The PCA involves a mathematical procedure that 
transforms a number of correlated variables into a number of 
uncorrected variables called principal components. The first 
principal component accounts for as much of the variance in 
the data as possible and each succeeding component accounts 
for as much of the remaining variance as possible. The second 
principal component is constrained to lie in the subspace 
perpendicular of the first. The third principal component is 
taken in the maximum variance direction in the subspace 
perpendicular to the first two and so on. The PCA is also 
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called as Karhunen-Love transform or the Hotelling transform 
[x]. 



Produce the cohunn vectors 
from input u^.^e; 



Ca.lcula.5e the co^ariance matrix 
o f tlie two c olumn v e ctors 
formed in 1 



Calculate the Eigen values and the 
Eigen vectors of the covariance 



NTc-rmali^e the column vecloi 



MomnaHzed Eigen vector act as the 
weight v alues which, multiply it 
with each pixel of the input images 



Fuse the two scaled matrices will be 
the fuse d image matnx . 



Figure 2: PCA Algorithm[x] 

F. Discrete Wavelet Transform (DWT) 
Wavelet transforms are mult i- resolution image decomposition 
tool that provide a variety of channels representing the image 
feature by different frequency sub -bands. It is a famous 
technique in analyzing signals. 2-D Discrete Wavelet 
Transformation (DWT) converts the image from the spatial 
domain to frequency domain. The image is divided by vertical 
and horizontal lines and represents the first -order of DWT, 
and the image can be separated with four parts those are LL1, 
LH1, HL1 and HH1 [i]. When decomposition is performed, 
the L-L band contains the average image information whereas 
the other bands contain directional information due to spatial 
orientation. Higher absolute values of wavelet coefficients in 
the high bands correspond to salient features such as edges or 
lines. In wavelet transformation due to sampling, the image 
size is halved in both spatial directions at each level of 
decomposition process thus leading to a multi resolution 
signal representation. The most important step for fusion is the 
formation of fusion pyramid. It is difficult to decide a uniform 
standard for fusion principle. 




Coeffkicr 





i 


■ 
J 




DWT 









Figure 3: DWT Transformation 
Fused Image Evaluation 
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Image quality metrics are used to benchmark different image 
processing algorithm by comparing the objective metrics. 
There are two types of metrics that is subjective and objective 
used to evaluate image quality. In subjective metric users rate 
the images based on the effect of degradation and it vary from 
user to user whereas objective quality metrics quantify the 
difference in the image due to processing. Furthermore, fusion 
assessment can be classified as either qualitative or 
quantitative in nature. Assessment of image fusion 
performance can be first divided into two categories [ix]: 
1) Assessment Without Reference Images: 

In assessment without reference images, the fused images 
are evaluated against the original source images for 
similarity. 

a) Entropy as a Quality Metric: Entropy is defined as 
amount of information contained in a signal. The 
entropy of the image can be evaluated as:- 



H = — Pii) log z (Pi d[)} 



b) 



....(4) 

Where G is the number of possible gray levels, P (di) 
is probability of occurrence of a particular gray level 
di. The maximum value of entropy can be produced 
when each gray level of the whole range has the same 
frequency. If entropy of fused image is higher than 
parent image then it indicates that the fused image 
contains more information. 

Standard Deviation as a Quality Metric: This metric 
is more efficient in the absence of noise. It measures 
the contrast in the fused image. An image with high 
contrast would have a high standard deviation. It is 
given by the following equation:- 



i = D 



c) 



....(5) 

Where hif (i) is the normalized histogram of the 
fused image, if (x, y) and L is number of frequency 
bins in histogram. 

Other parameters are: Cross entropy, Fusion mutual 
information, Fusion quality index, Fusion similarity 
metric, Spatial Frequency etc. 



2) Assessment With Reference Image : 

In reference-based assessment, a fused image is evaluated 

against the reference image which serves as a ground 

truth. 

a) Root Mean Square Error (RMSE): A commonly used 
reference-based assessment metric is the root mean 
square error (RMSE) which is defined as folio ws:- 



RMSE = LnLLw- Bi # 



....(6) 



IJSET@2014 



Page 377 



IJ5ET 



b) 



c) 



d) 



e) 



f) 
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Where, A is the perfect image, B is the fused image 
to be assessed, i is pixel row index, j is pixel column 
index and m, n are the no. of rows and columns. 
Mean Squared Error (MSE): The mathematical 
equation of MSE is given by the following 
equation:- 



1 m a 

MSE = y Y (Aij - Bij f 

mn i—i i—i 

....(7) 

Normalized Cross Correlation (NCC): Normalized 
cross correlation are used to find out similarities 
between fused image and registered image is given 
by the following equation[l]:- 



NCC = 



....(8) 

Peak Signal to Noise Ratio (PSNR): PSNR is the 
ratio between the maximum possible power of a 
signal and the power of corrupting noise that affects 
the fidelity of its representation [1]. The PSNR 
measure is given by:- 



PNSR = 20 log 



....(9) 

Structural Content (SC): The structural content 
measure is used to compare two images in a number 
of small image patches the images have in common. 
The large value of structural content SC means that 
image has poor quality. This value should be small. 
The SC measure is given by:- 



SC = 



....(10) 

Other parameters are: Percentage fit error, Signal to 
noise ratio, Mutual information, Universal quality 
index, Measure of structural similarity etc. 

Comparison Of Different Image Fusion 
Techniques 



Table (1): Advantages and Disadvantages of the Different 
Image Fusion Techniques [iv]. 



Sr. 
No. 


Fusion 
Method 


Advantages 


Disadvantages 


1 


Simple 
Average 


Simplest, works 
well for single 
sensor images 
containing additive 
noise, good for 
high contrast input 
images 


Reduced contrast 
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Simple 

Max/ 

Min 



Brovey 
Method 



IHS 



PCA 



DWT 



Simple, highly 
focused output 
image obtained as 
compared to 
average method. 



Simple and fast 
method, good for 
multisensory 
images, provides 
superior visual and 
high resolution MS 
image. 



It is a simple 
method to merge 
the images 
attributes and 
provides 

high spatial quality 



Simple, fused 
images have high 
spatial quality. It 
prevents certain 
features from 
dominating the 
image because of 
their large digital 
numbers. 



The DWT fusion 
method has least 
spectral distortion. 
It also provides 
better SNR than 
pixel based 
approach. 



Blurring, it has higher 
pixel intensity but it 
does not mean more 
content. 



This method ignores 
the requirement of 
high quality synthesis 
of spectral 
information and 
causes spectral 
distortion 



It suffers from 
artifacts and noise 
which tends to higher 
contrast. It causes 
colour distortion. The 
major limitation that 
only three bands are 
involved 



This method is highly 
criticized because of 
the distortion of the 
spectral characteristic 
between the fused 
images and the 
original low 
resolution images [v]. 
Resulting image does 
not preserve faithfully 
the colors found in 
the original images 



In this method final 
fused image have a 
less spatial resolution. 



VIL Applications 

Some of the applications of Image Fusion Techniques in given 
below: - 

1. Intelligent robots 

a. Require motion control, based on feedback from the 
environment from visual, tactile, force/torque, and 
other types of sensors. 

b. Stereo camera fusion. 

c. Intelligent viewing control. 

d. Automatic target recognition and tracking. 

2. Medical image 

a. Fusing X-ray computed tomography (CT) and 
magnetic resonance (MR) images. 

b. Computer assisted surgery. 

c. Spatial registration of 3-D surface. 
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3. Manufacturing 

a. Electronic circuit and component inspection. 

b. Product surface measurement and inspection. 

c. Non-destructive material inspection. 

d. Manufacture process monitoring. 

e. Complex machine/device diagnostics. 

f. Intelligent robots on assembly lines. 

4. Military and law enforcement 

a. Detection, tracking, identification of ocean (air, 
ground) target/ event. 

b. Concealed weapon detection. 

c. Battle-field monitoring. 

d. Night pilot guidance. 

5. Remote sensing etc. 

VIII. Conclusion 

This report performs the Comparative study of Image fusion 
techniques. Here, various Image Fusion techniques that are 
useful to create a single enhanced image more suitable for the 
purpose of human visual perception, object detection and 
target recognition have been discussed with applications. On 
the basis of the study and examples mentioned we have found 
out various issues in different techniques. This review 
concludes which approach is better among all the existing 
Image Fusion techniques. Although selection of fusion 
algorithm is problem dependent but this review concludes that 
spatial domain techniques provide high spatial resolution but 
these techniques have image blurring problem. The Wavelet 
transform is a very good technique for image fusion providing 
a high quality spectral content. 
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